Sequential filtering for multi-frame visual reconstruction

Al~a'aet. We describe an extension of the single-frame visual field reconstruction problem in which we consider how to efficiently and optimally fuse multiple frames of measurements obtained from images arriving sequentially over time. Specifically we extend the notion of spatial coherence constraints, used to regularize single-frame problems, to the time axis yielding temporal coherence constraints. An information form variant of the Kalman filter is presented which yields the optimal maximum likelihood estimate of the field at each time instant and is tailored to the visual field reconstruction problem. Propagation and even storage of the optimal information matrices for visual problems is prohibitive, however, since their size is on the order of 104 x 104 to 106 x 106. To cope with this dimensionality problem a practical yet near-optimal filter is presented. The key to this solution is the observation that the information matrix, i.e. the inverse of the covariance matrix, of a vector of samples of a spatially distributed process may be precisely interpreted as specifying a Markov random field model for the estimation error process. This insight leads directly to the idea of obtaining low-order approximate models for the estimation error in a recursive filter through the recursive approximation of the information matrix by an appropriate sparse, spatially localized matrix. Numerical experiments are presented to demonstrate the efficacy of the proposed filter and approximations.